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ABSTRACT 



This manual provides a beginning approach for research 
libraries to better describe the use and users of their networked services. 
The mamual also aims to increase the visibility and importance of developing 
such statistics and measures. Specific objectives are: to identify selected 
key statistics and measures that can describe use and users of electronic and 
networked services; to standardize procedures and definitions to collect 
these statistics and measures; and to increase awareness of selected issues 
related to collecting, analyzing, and reporting the data to produce these 
statistics and measures. Definition, rationale, implementation, special 
considerations, and related issues are described for network statistics in 
the following categories: patron accessible electronic resources; use of 
networked resources and services; expenditures for networked resources and 
related infrastructure; and library digitization activities. Definition, 
rationale, implementation, special considerations, and related issues are 
also described for performance measures. Appendices include a list of 
statistics considered and forms for data collection. (MES) 
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Introduction 



There is a critical need for academic and research libraries to develop new 
statistics and measures to describe network services and resources. This manual is 
one product of a larger New Measures Initiative by the Association of Research 
Libraries (ARL) <http://www.arl.org/stats/newmeas/newmeas.html>. The E- 
Metrics project that developed this manual is funded by a group of 24 ARL mem- 
ber libraries. 







Based on a substantial field-testing process (described in detail in the E- 
Metrics Phase II report <http://www.arl.org/stats/newmeas/emetrics/>), the study 
team recommends a number of network statistics and performance measures that 
provide indicators of library networked services and resources. 

For a host of reasons not explored in this manual, the final set of statistics and 
measures were selected among others that were considered but not discussed in detail herein. 
The selection of certain measures over others is not meant to imply that those not selected 
are flawed or have no beneficial use. See Appendix A for a table containing three lists of 
candidate measures considered during the E-Metrics project, and refer to Part 2 of the E- 
Metrics Phase II Report for a discussion of the evolution of measures. 

These statistics and measures will provide research libraries with an impor- 
tant and useful set of tools to describe and assess network resources and services. 
The manual also provides libraries with guidance regarding the use to which the 
network statistics and measures can be put. 

The manual offered here has a number of specific goals and objectives. Its 
primary goal is to provide a beginning approach for research libraries to better 
describe the use and users of their networked services. A secondary goal is to 
increase the visibility and importance of developing such statistics and measures. 
Specific objectives of the manual are to: 

□ Identify selected key statistics and measures that can describe use and 
users of electronic and networked services; 

□ Standardize procedures and definitions to collect these statistics and 
measures; and 

□ Increase awareness of selected issues related to collecting, analyzing, and 
reporting the data to produce these statistics and measures. 

The statistics and measures offered here will need to be continually developed, 
expanded, refined, and possibly eliminated over time. 

A key component of the project has been to work with vendors and other 
organizations regarding the collection, manipulation, and reporting of vendor- 
supplied online database data. Many of the statistics described here resulted from 
the cooperative efforts among these vendors and other national/international 
groups interested in developing such statistics. Such efforts should be continued. 

Given the rapidly changing technology environment, the changing milieu of 
higher education, changing organizational structures within ARL libraries, and the 
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complexity of measuring such networked services, it is almost certain that the 
statistics and measures proposed in this manual will continue to evolve. These 
measurement tools, however, will provide research librarians with important 
techniques to count, describe, and report networked services and resources in their 
libraries. 
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Preface 



This manual provides definitions, data collection 
procedures, and discusses related issues pertaining to in- 
terpreting and using the recommended statistics and mea- 
sures. The definitions and procedures vrere derived from 
a month of field-testing at more than a dozen ARL librar- 
ies. The statistics and performance measures represent a 
minimum set of data that need to be collected continually 
and used. Individual libraries will need to develop local 
procedures to support data collection activities within the 
guidelines of this manual. However, readers need to rec- 
ognize that the statistics and measures will be refined and 
extended continually in the future. PowerPoint instruc- 
tional modules to accompany this manual are available 
from ARL. 



Data Collection Manual for Academic 
and Research LibraiiNetworkStatistics 
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Recommended Statistics and Measures 



Network Statistics 

Patron Accessible Electronic Resources 

□ R1 Number of electronic full-text journals (p. 5) 

□ R2 Number of electronic reference sources (p. 7) 

□ R3 Number of electronic books (p. 8) 

Use of Networked Resources and Services 

□ Ul Number of electronic reference transactions (p. 11) 

□ U2 Number of logins (sessions) to electronic databases (p. 13) 

□ U3 Number of queries (searches) in electronic databases (p. 14) 

□ U4 Items requested in electronic database (p. 15) 

□ U5 Virtual visits to library’s website and catalog (p. 16) 

Expenditures for Networked Resources and Related Infrastructure 

□ Cl Cost of electronic full-text journals (p. 20) 

□ C2 Cost of electronic reference sources (p. 21) 

□ C3 Cost of electronic books (p. 22) 

□ C4 Library expenditures for bibliographic utilities, networks, and consortia (p. 24) 

□ C5 External expenditures for bibliographic utilities, networks, and consortia (p. 25) 

Library Digitization Activities 

□ Dl Size of library digital collection (p. 27) 

□ D2 Use of library digital collection (p. 29) 

□ D3 Cost of digital collection construction and management (p. 30) 

Performance Measures 

Performance Measures 

□ PI Percentage of electronic reference transactions of total reference (p. 33) 

□ P2 Percentage of virtual library visits of all library visits (p. 34) 

□ P3 Percentage of electronic books to all monographs (p. 35) 
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Criteria for Performance Statistics and Measures 



T he data collection procedures for the statistics and performance measures 
are defined and described according to the following criteria: 

Definition: Describes each statistic or performance measure. 

Rationale: Discusses why the suggested statistic or performance measure is 
needed and/or how it can be useful to describe electronic resources and 
services. 
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Implementation: Provides instructions for implementing the 

identified statistic or performance measure, categorized by collector, 
frequency, procedures, and special considerations, if any. 

Collected by: Identifies who is responsible for collecting data; 

/oca/ refers to the individual library and vendors refers to the content 
providers with whom the library has contracted to provide electronic 
resources. 

Frequency: Identifies how often the statistic/measure needs to be 

collected. 

Procedures: Outlines the manner in which the data for a statistic 

or performance measure may be collected. Also includes recommendations 
for forms. 

Special considerations: Identifies special factors that need to be 
considered during data collection or interpretation. 

Related issues: Discusses issues that go beyond the suggested 

data collection procedures, such as the availability of complementary 
statistics, ways in which statistics can be combined with other statistics, 
and other possible approaches to data collection. 
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Data Collection Procedures for Performance Statistics 

and Measures 






Statistics Related to Patron Accessible Resources 




T he statistics developed for patron accessible resources account for 
networked resources and services. The current ARL membership 
criteria index lacks separate measures for electronic and networked 
monographs, serials, and bibliographic utilities. Though these electronic and 
networked resources may limit the amount of print materials acquired and may cost 
more than their print counterparts, they do constitute more widely available 
resources. 




© 



In the electronic and networked realm, the more a library has, the more 
materials are provided to customers anytime and anywhere. Although local needs 
and available resource allocations may differ from library to library, the resource 
statistics allow academic research libraries to see and to demonstrate to others the 
changing nature of library collections over the years. In turn, the libraries are 
expected to use them to make decisions about resource allocations (budget, staff, 
time, etc.) and to undertake strategic planning accordingly. Furthermore, the 
picture of available resources provides libraries with an opportunity to offer valued 
services. However, because the evolving nature of these statistics will rely heavily 
on technological enhancements, all libraries are encouraged to use extra caution 
while serving their institutional goals, missions, and visions. 



n Rl Number of electronic full-text 
journals 

□ R2 Number of electronic reference 
sources 

□ R3 Number of electronic books 



Patron Accessible 
Electronic 
Resources 






Number of electronic full-text 
Journals 




Definition: Number of electronic full-text journal subscriptions that 
the library provides to users either through an individual institutional licensing 
contract with the provider of journals or through other arrangements (e.g., regional 
or state consortium) for which the library pays a reduced or no fee for access. 

The full-text journals should provide both search and browse capabilities by 
title and issue. This is different from journal article databases, such as Expanded 
Academic ASAP in INFOTRAC, that do not provide browsing capability. 
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This includes electronic full-text journals offered by established scholarly 
journal publishing houses (e.g., Elsevier’s ScienceDirect and Academic Press’s 
IDEAL), scholarly societies (e.g., American Chemical Society journals and Ameri- 
can Institute of Physics Online), and services which aggregate content from 
smaller publishers or from those publishers that prefer to use an external delivery 
platform (Highwire, OCLC ECO, and EbscoOnline). This should exclude gen- 
eral-purpose periodicals such as magazines and newspapers. 

Rationale: Electronic access has expanded dramatically to provide a 

range of useful resources for library users. This statistic helps document the 
degree of expansion of electronic resource availability in the individual library and 
can be used to justify continuation and enhancement of these services. 

Research libraries act increasingly as gateways to a vast array of external 
information. This measure specifically addresses the extensiveness of scholarly 
content a library provides to its user community. In many cases, electronic access 
enables the library to offer larger selections of journals than it could provide in 
paper format. This statistic can also be used for library promotion and internal 
and external reporting. Particularly, this statistic aims at showing the changing 
nature of traditional scholarly resources with improved and better access anytime 
and anywhere. 

Implementation 

□ Collected by: Local and vendors 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly) 

□ Procedures: It is impossible to obtain the complete list of electronic full-text 
journals from a single source. Possible sources for the information include 
library catalog records (those records that point to web addresses), library web 
pages that list the journal titles, the internal electronic resource management 
database, and vendor records (websites and contract documentation). 

1. Create a master list of full-text electronic journals from all the sources 
available. Use a spreadsheet or database program to organize and maintain 
the list. 

2. Remove titles that do not meet the above-mentioned definition but keep 
duplicate titles resulting from multiple subscriptions (e.g., main library 
and medical or law libraries). 

3. Record the counts and be sure they are updated regularly. The library 
should also update the count information on the library website and/or in 
marketing brochures on a regular basis. 

Special considerations: It is time-consuming to establish procedures 
to collect this statistic for the first time. However, once that is done, it will be 
relatively easy to update the information. (This applies to other statistics and 
measures included in the manual as well.) 

Include journal titles that come with print subscriptions or print plus online 
subscriptions since the focus of the statistic has to do with how many scholarly 
electronic journal titles users can access. Do not include free government publica- 
tions and free electronic journals to which the library provides links. 
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Free government publications and free electronic journals are a valuable 
resource for many libraries. How to collect statistics relating to these resources will 
be addressed in the future. 




Number of electronic reference sources 






Definition: Number of electronic reference sources and aggregation 
services that the library provides to users either through an individual licensing 
contract with the content providers or through other arrangements (e.g., regional 
or state consortium) for which the library pays a reduced or no fee for access. 

This includes citation indexes and abstracts; full-text reference sources (e.g. 
encyclopedias, almanacs, biographical and statistical sources, and other quick fact- 
finding sources); full-text journal and periodical article collection services (e.g., 
EBSCOhost, ProQuest, Academic Universe, and INFOTRAC OneFile); dissertation 
and conference proceedings databases; and general-purpose magazines and newspa- 
pers. Licensed electronic resources also include those databases that institutions 
mount locally 

Rationale: Networking technology in libraries has improved and 
increased dramatically user access to a range of useful reference resources. This 
statistic documents the degree of expansion of electronic resource availability and 
can be used to justify continuation and enhancement of these services. In the 
1990s, because of the increasing popularity of the Internet, the ways reference 
interviews were held and reference sources were used changed. Today, users have 
electronic formats as well as traditional reference sources to provide answers to 
their reference questions. 

Research libraries traditionally act as gateways to a vast array of external 
information. This measure deals with the extensiveness of scholarly content the 
library provides to the user community and the availability of reference sources on 
an anytime/anywhere basis. In many cases, electronic access enables the library to 
offer more resources than it could in paper format. This statistic can also be used 
for library promotion and internal and external reporting. Specifically, this statistic 
aims at showing the changing nature of traditional scholarly resources with im- 
proved access. 

Implementation 

□ Collected by: Local and vendors 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly) 

□ Procedures: As in the case of the number of electronic full-text journals, it is 
impossible to obtain the complete list of databases from a single source. Pos- 
sible sources for the information include library catalog records (those records 
that point to web addresses), library web pages that list the database titles, the 
internal electronic resource management database, and vendor records (websites 
and contract documentation). 
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1. Create a master list of electronic databases from all the sources available. 
Use a spreadsheet or database program to organize and maintain the list. 

2. Remove titles that do not meet the above-mentioned definition but keep 
duplicate titles resulting from multiple subscriptions (e.g., main library and 
medical or law libraries). 

3. Record the counts and be sure they are updated regularly. The library 
should also update the count information on the library website and/or in 
marketing brochures on a regular basis. 

Special considerations: The unit of measurement here is the data- 
base not the whole service provided by a vendor. For example, if the library 
subscribes to OVID and the company provides five databases (ABI/Inform, Books 
in Print, CINAHL, INSPEC, and PsycINFO), then the count is 5, not 1. By the 
same token, if the library subscribes to three database packages (Academic Uni- 
verse, Congressional Universe, and Statistical Universe) from Lexis-Nexis, the 
count is 3. 

This count should not include freely available databases to which the library 
provides links or library-created finding aids. 

Freely available databases and library-created finding aids are a valuable 
resource for many libraries. How to collect statistics relating to these resources will 
be addressed in the future. 

\ 

Number of Electronic Books 

Definition: Number of electronic full-text monographs that the 
library offers to its users either through an individual licensing contract with the 
content providers or through other arrangements (e.g., regional or state consor- 
tium) where the library pays a reduced or no fee for access. 

This includes electronic books purchased through vendors, such as netLibrary 
and Books24x7, and electronic books that come as part of aggregate services. It 
excludes internally digitized electronic books, electronic theses and dissertations, 
digitally created archival collections (e.g.. Early English Books Online), and other 
special collections. This also excludes publicly available electronic books to which 
the library provides web links. It does not include machine-readable books distrib- 
uted on CD-ROM, or accompanied by print books. 

Rationale: Networking technology in libraries has improved and 
increased dramatically user access to the electronic counterparts of some traditional 
sources. This statistic documents the degree of expansion of e-books. In the mid 
90s, networking and resource sharing technologies provided libraries with print 
books and e-books that were made available through a library’s networks. 

Because the evolving nature of this statistic will heavily depend on techno- 
logical enhancements, all libraries are encouraged to use extra caution while 
pursuing their institutional goals, missions, and visions. Moreover, the definition 
of e-books is still evolving. This statistic is an early attempt to keep track of this 
type of resource as it becomes more widely available. 
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Implementation 

O Collected by: Local and vendors 

O Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly) 

O Procedures: 

1. For each electronic book collection, get the electronic title counts from 
either the providers or catalog records. Unlike electronic full-text journals 
and reference databases, it is not necessary to list the titles for each elec- 
tronic book collection. 

2. Count any duplicate titles resulting from multiple subscriptions (e.g., main 
library and medical or law libraries). 

3. Record the counts and be sure they are updated regularly. The library 
should also update the count information on the library website and/or in 
marketing brochures on a regular basis. 

Special considerations; Do not include book collections that are a 
part of aggregate services and function more as a reference collection (e.g., MD 
Consult reference books, ProQuest's Early English Books Online, and 
books@OVlD). They should be reported in the electronic reference databases. 

Do not include freely available electronic books such as titles available from 
the National Academy Press. 

Related issues; Electronic books, still evolving in terms of technol- 
ogy and adoption for use, present a number of issues in terms of definition and 
measurement, such as “location,” accessibility (metadata and access points), and use 
versus circulation (e.g., is online use for 20 minutes a circulation, as it would be 
with reserve materials, or does a circulation of electronic books require a minimum 
period of use, such as 24 hours?). 

□ What about reference book collections provided by vendors? Should they 
be treated as electronic books, for example, or should they be treated as a 
database, on the grounds that they are used as databases? 

Count only those books that a user can check out, as they would traditional 
books. Unlike traditional books that the library purchases and owns, electronic 
books can be subscribed to for an ongoing fee. In this case, the library accounting 
system may treat these as serials rather than books because of the type of payment. 
It is relatively easy to keep track of the number of electronic books right now since 
most libraries deal with only a handful of e-book vendors, such as netLibrary and 
Books24x7. But in the future, it will become increasingly difficult to do this as the 
sources of electronic books proliferate. 

Finally, some provisions of contractual agreements between libraries and 
vendors may limit the level of use of e-books. These issues need be addressed in 
future research. 
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Dse of Electronic Networked Resources and Services 






H igh use of a library resource or service implies a collection development 
program that is working to create access to the resources customers 
need. Use and the need can also identify resources and services that are seen as 
particularly valuable in the education and research enterprise and should be ex- 
panded, or perhaps resources and services that should be discontinued due to lack 
of use and interest. Whether provided by vendors or collected institutionally, usage 
statistics can help a library administrator make decisions and plan for the future in 
order to meet not only users’ expectations and needs but also institutional goals. 
The reported data can also provide other information as to where and when people 
use the library’s materials and how well the library serves its target audience and 
anticipates their potential needs. 

The cost of providing access to networked resources and services can be more 
expensive than that of traditional counterparts. Depending heavily on earlier 
ICOLC guidelines, the E-Metrics use measures put this in the perspective of the 
changing academic research library environment. The purpose of the use measures 
is to provide statistics relating to the use of networked services and resources. 
Therefore, it is expected that library administrators can reconsider some resource 
allocation issues as the number of resources and services tend to increase while 
people are provided greater access. Please note that, as with most of the statistics 
in this study, statistics related to the use of library resources and services should be 
revisited and perhaps modified as the technology advances. 
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n Ul Number of electronic reference 
transactions 

□ U2 Number of logins (sessions) to 
electronic databases 

□ U3 Number of queries (searches) in 
electronic databases 

n U4 Items requested in electronic database 
n U5 Virtual visits to library’s website and 
catalog 



Use of Networked Resources 
and Services 
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Number of electronic reference 

TRANSACTIONS 
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Definition: Number of electronic reference transactions conducted 

via email, a library’s website, or other network communications mechanisms , — . 

designed to support electronic reference. An electronic reference transaction must 
include a question either received electronically (e.g., via e-mail, WWW form, etc.) (fjs) 
or responded to electronically. Those transactions that are both received and 
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responded to electronically are counted as one transaction. This count excludes 
phone and fax traffic unless either the question or answer transaction occurs via the 
described manner. It includes the counts accrued from participation in any local 
and national projects, such as DigiRef and the Library of Congresses CDRS (Col- 
laborative Digital Reference Service). 

A reference transaction is an information contact, which involves the knowl- 
edge, use, recommendations, interpretation, or instruction in the use of one or 
more information sources by a member of the library staff. 

Rationale: Libraries are making more of their services available 
electronically and are interested in tracking the development of a new and emerg- 
ing library service. There is a need to better document this transition to facilitate 
and improve resource allocation activities. This statistic represents reference 
activities conducted electronically in the library. It is an attempt to measure 
reference transactions through new electronic tools. 

Implementation 

□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(monthly, quarterly). This statistic can be collected in the same manner as the 
library gathers other reference transactions data. 

□ Procedures: 

1. Select a typical week (or month) to run a sample study. Be sure to vary the 
specific week (or month) chosen over the course of a year or from year to 
year to account for seasonal fluctuations. 

2. Key tasks include distributing a daily tally sheet, collecting the daily tally 
sheet, adding each day^s totals to a weekly figure, and being available to 
respond to data collection problems should they occur. 

3. Transactions may be via e-mail, a form on a web page, etc. Electronic 
reference transactions may involve more than reference desk staff (e.g., web 
master, various reference personnel, library director, volunteers, etc.). 
Establish an administrative procedure to report electronic reference trans- 
action counts to a designated staff person, no matter who receives the 
questions or answers the reference requests. 

4. Disseminate the new procedure and rationale. Several notices throughout 
the year may be necessary. 

5. Report an electronic reference transaction as you would a face-to-face 
reference transaction. Thus, one e-mail request may contain several 
reference questions taking varying times to complete. For example, one e- 
mail request could contain two relatively short reference questions and one 
reference question that took 10-15 minutes to answer. Count the number 
of requests, not the number of questions. Thus, in the example you would 
report one (1) as the number of electronic reference transactions even 
though there were three questions. Report counts using pre-established 
local library reporting periods (weekly, monthly, etc.). 

6. Indicate and describe any additional methods used outside of this defini- 
tion and guidelines. 



Special considerations: Unless the library uses electronic reference 
management software to collect and report transaction data, it is difficult to keep 
track of a complete reference transaction cycle (query and response) because of 
time-delays and the involvement of several parties. 

As stated in the definition, the statistic includes the number of service 
transactions provided to patrons outside the university or the parent institution 
that the library serves, through regional or national cooperative efforts and through 
library policies. 

Related issues: Reference services are undergoing rapid changes. 
Libraries are experimenting with different modes of electronic reference. One 
could say that simple email transactions that are prominently mentioned in the 
procedures are not much different from traditional reference services. How can a 
library measure quality in providing different types of electronic reference services 
such as live-chat with text/voice/video? Will this measure help the library deter- 
mine user demand and thereby plan for resource allocation? To answer these 
questions, libraries need to collect more detailed information such as length of time 
taken to answer questions, types of questions by types of transactions, and so on. 
Also, this statistic is likely to produce some useful figures and trends regarding 
staff support and allocation in reference activities. 




Number of logins (sessions) to electronic 

DATABASES 
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Definition: Number of user initiated sessions in licensed electronic 

resources. A session or login is one cycle of user activities that typically starts 
when a user connects to a database and ends with explicit termination of activities 
(by leaving the database through logout or exit) or implicit termination (time out 
due to user inactivity). Licensed electronic resources also include those databases 
that institutions mount locally. 

Rationale: One purpose of having a networked environment is to 
promote connectedness and accessibility to a variety of information resources, 
hence the need for this measure. Also, the gradual shift in the materials expendi- 
tures from traditional print-based resources to electronic databases can be under- 
stood with the measure. This measure will produce a count of how often specific 
databases are used and complement traditional physical attendance counts. 

Implementation: Until there is a standardized report generation system 
that captures different statistics from different content providers, we recommend 
that each library develop an in-house spreadsheet or database to capture monthly 
usage statistics of licensed databases. At least on the database title level, usage 
statistics should be collected from vendors, entered into the in-house databank, and 
maintained for reporting and analysis. 








© 
i < D 



□ Collected by: Vendor 

□ Frequency: Monthly, but can be reported quarterly or annually 

□ Procedures: ^ 
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1. Process monthly usage statistics from vendors and copy or import the 
number of attempted sessions in each database (in each journal collection 
for full-text journals) to an in-house spreadsheet or database file. 

2. Calculate the total sessions for a given month by adding the number of 
sessions from each database or journal collection. 

Sp ecial considerations: Not all vendors report this statistic. There- 

fore, it will be necessary to qualify the statistic with a sentence such as this: “We 
have 150,000 logins recorded from 120 databases out of 200 subscribing. We 
cannot report this statistic for the remaining 80 databases because the vendor does 
not supply login (session) information to customers.” 

Related issues: When analyzing the login counts, it might be impor- 
tant to explain any increases or decreases in the figures. Specify, for example, 
whether the increase comes from (1) the addition of new databases, (2) databases 
which did not report the statistic in the past but have now begun reporting, (3) 
increased demand, and/or (4) an increase in the number of simultaneous users. 

Problems with the comparability of login counts from different vendors is a 
serious threat to the utility of the combined count. Content providers use different 
time-out thresholds (ranging from 7 to 30 minutes on average). Also, because of 
the IP-based authentication, several sessions conducted at the same public work- 
station can be counted as a single login. Alternatively, libraries can collect at- 
tempted logins to various licensed databases by making users go through a central 
gateway (which counts all attempted logins). This will ensure that one login 
attempt to a database is the same as a login to other databases. However, what this 
data collection method misses is user logins that go directly to content provider 
sites. It is unclear how many user logins fall into this category, but the phenom- 
enon certainly results in a substantial undercount of user logins. 

While the gross login figure is useful, it is useful only for trend plotting and 
gross justification of electronic resources. Within the library, the usage measures 
of licensed electronic resources have many users and uses. Circulation of usage 
statistics on the database title level (or in an extreme case on the journal title level) 
and discussion of any noticeable changes (or lack thereof) need to occur at various 
levels among the concerned parties, including collection development personnel, 
web master(s), technical services staff, and so on. 
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Number of queries (searches) in Electronic 
Databases 
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Definition: Number of user initiated queries (searches) in licensed 
electronic resources. A search is intended to represent a unique intellectual in- 
quiry. Typically, a search is recorded each time a search request is sent/submitted 
to the server. 



© 



Rationale: This statistic provides libraries with an indication of the 
databases that are most heavily used, areas of user interest, database popularity, and 
a level of usage detail that goes beyond an initial session. It also can provide 






important information for billing purposes, as some vendors charge for database 
usage by number of searches. This statistic can complement Ul, the number of 
electronic reference transactions, as more user requests bypass staff mediations. 
Some portion of this statistic is also analogous to in-library use of reference 
sources. 

Implementation 

□ Collected by: Vendor 

□ Frequency: Monthly, but can be reported quarterly or annually 

□ Procedures: Until there is a standardized report generation system that captures 
different statistics from different content providers, we recommend that each 
library develop an in-house spreadsheet or database to capture monthly usage 
statistics of licensed databases. Usage statistics need to be collected from 
vendors, entered into the in-house databank, and maintained for reporting and 
analysis. 

1. Process monthly usage statistics from vendors and copy or import the 
number of attempted searches in each database to an in-house spreadsheet 
or database file. 

2. Calculate the total number of searches for a given month by adding the 
number of searches from each database or journal collection. 

Special considerations: Because some vendors do not report this 
statistic, it will be necessary to qualify the statistic with a sentence such as this: 
“We have 150,000 searches recorded from 120 databases out of 200 subscribing. 
The other 80 do not provide this statistic.” 

Related issues: Different assumptions about and mechanisms for 
collecting search counts by different vendors are potential threats to the combined 
count. 




Items requested in electronic databases 



Definition: Number of items requested in all of the library’s licensed 
electronic resources. These resources may include journal articles, e-books, refer- 
ence materials, and non-textual resources that are provided to the library’s users 
through licensing and contractual agreements. The user requests may include 
viewing, downloading, emailing, and printing to the extent the activity can be 
recorded and controlled by the server rather than browser. 

The items reported depend on the type of content. Examples include cita- 
tions, abstracts, tables of contents, and full-text articles (ASCII, HTML, PDF, or 
PS). 

Rationale: This statistic provides a circulation count for electronic 
contents in a way analogous to the traditional circulation of books. Given the fact 
that libraries do not have good measurements of in-house materials usage, particu- 
larly serials usage, this statistic helps libraries understand in-library use patterns 
that were heretofore difficult to measure. 
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Implementation 

□ Collected by: Vendor 

□ Frequency: Monthly, but can be reported quarterly or annually 

□ Procedures: Until there is a standardized report generation system that captures 
different statistics from different content providers, we recommend that each 
library develop an in-house spreadsheet or database to capture monthly usage 
statistics of licensed databases. Usage statistics should be collected from ven- 
dors, entered into the in-house databank, and maintained for reporting and 
analysis. 

1. Process monthly usage statistics from vendors and copy or import the 
number of items selected for viewing, downloading, and emailing in each 
database. Count the number and type of items users selected: abstracts, 
citations, and full-texts. 

2. Calculate the total number of items for a given month by adding the 
number of items requested from each database or journal collection. 

Special considerations: Because some vendors do not report this 
statistic, it will be necessary to qualify the statistic with a sentence such as this: 
“More than 150,000 items were requested from 120 databases out of 200 subscrib- 
ing. The other 80 do not provide this statistic. Among the requested items, 
100,000 were some form of full-text records.” 

Related issues: Different vendors apply different assumptions and 

mechanisms in collecting items requested counts. This lack of standardization 
makes it difficult to calculate an aggregate count. 

We do not have good measurement of in-house materials usage, particularly 
journal usage. However, electronic journals and databases allow libraries to find 
out how often materials are requested. Having in-house usage figures is important 
for understanding the dynamics of usage between print and electronic journals, so 
that we can ascertain any correlation between them. 




Virtual visits to library's website and catalog 



Definition: This is defined as user visits to the library’s website or 
catalog from outside the physical library premises regardless of the number of 
pages or elements viewed. If a user looks at 16 pages and 54 graphic images while 
at a website, that user registers one visit on the web server. All visits to the website 
should be counted regardless of repetition by one user. A visit is usually deter- 
mined by a user’s IP address, which can be misleading due to Internet Service 
Providers (ISPs) and Firewalls or Proxy Servers. Thus, this measure is actually an 
estimate of the visits. 

Rationale: Use of the website or catalog from outside the library 
reflects interest in library services. The role of networked services is to expand the 
reach of libraries beyond their physical boundaries. This statistic helps describe 
the significance of networked services use by measuring the number of virtual 



accesses. This will also give an opportunity for the library to compare the demand 
placed on their networked resources with that for other popular information- 
oriented websites (such as Excite, Lycos, etc.). 

Implementation: 

□ Collected by: Local 

n Frequency: Reported annually, although internal reporting will be more frequent 
(e.g., weekly, monthly, and quarterly). 

□ Procedures: 

1. Identify all sources of virtual visits to the library. This may involve activi- 
ties that take place on more than one web server. Some of the web servers 
may be owned by the library and some may be owned or maintained by 
another department in the university, an Internet Service Provider (ISP), or 
other library vendors (e.g., library OPAC provider). 

2. Exclude internal use within the premises of the library from the counts for 
this measure when possible. Two common approaches are using IP ad- 
dresses or some form of authentication tagged to each transaction. In 
terms of external visits to the library, three common sources are: external 
access to the library’s web page, remote logins (sessions) to non-web-based 
library databases, and remotely accessible library OPAC. 

3. Develop strategies for collecting the data from each of these sources of 
virtual visits. Different software may be needed to measure each electronic 
source of virtual visits. In some cases, the library may calculate the virtual 
visits using one or more log analysis software packages. In other cases, the 
external owner of the web server or service (the ISP) must provide the data. 
Discussions may need to be held with these service providers to obtain the 
needed data. In still other cases, custom programs may have to be devel- 
oped. 

4. In the case of library web pages housed on the library server, identify, 
configure, and install appropriate log analysis software. Determine log 
analysis software definition that corresponds to the virtual visit definition. 

Note: Different log analysis software packages may count virtual visits in 
different ways, so the count obtained will by necessity be an estimate. 
Arrange with the server technical staff for regular (monthly) reporting of 
internal visits at the various user access Internet workstations, external 
library user virtual visits, and total virtual visits (internal visits plus exter- 
nal visits). Run the log analysis software. 

5. In the case of library web pages housed on an ISP’s server, identify the log 
analysis software the ISP uses. Determine the definition of “visit” used by 
the log analysis software that corresponds to the virtual visit definition 
with the assistance of the ISP. Arrange with the ISP for regular (monthly) 
reporting of internal library visits at the various user access Internet 
workstations, external library user virtual visits, and total virtual visits 
(internal visits plus external visits). 

6. Where virtual visit counts include the aggregate of internal and external 
visits, indicate this in your report. 
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Special considerations: Count all visits to the website regardless of 
repetition by one user as long as each visit meets the criteria for this statistic. 

After one user connects to the Internet, several users could conduct multiple 
different searches in the electronic service. In some cases, e.g., Internet-accessible 
OPAC use inside the library, several users, one after the other, might make use of 
the same established connection. In most systems, a connection is cut off after a 
specified period of non-use, thus solving part of the problem. The best existing 
method of collecting virtual visits is to use log analysis software. The log analysis 
software producers may define virtual visits differently. For example, does a visit 
end after a time-out period of 30 minutes, 15 minutes, or some other time? The 
recommended time-out period is 30 minutes, but a local library may have to accept 
the available log analysis software’s definition even if it varies from the above. 

Some libraries will find it difficult to report every virtual visit. For example, 
libraries may have difficulty counting the use of library OPACS because their 
vendors do not provide this information. Make a record of those sources of virtual 
visits not counted. Do not estimate virtual visits for which data are not available. 

Related issues: This measurement requires a relatively high degree of 

technical skills either on staff or available from the library’s website host. 
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Expenditures for Electronic Resources and Related 

Infrastructure 



T his portion of the statistics is based on the ARL Supplementary Statistics 
Survey (the most recent survey instruction available at http:// 
www.arl.org/stats/arlstat/#sup h In collecting the statistics, the library should refer 
to the procedures followed and the amounts reported in response to the ARL 
Supplementary Statistics Survey. 

These statistics were developed by ARL to determine expenditure patterns on 
electronic and networked resources and the effect of new types of library resources 
and services, those delivered both individually and collectively with other institu- 
tions, on library expenditures. These measures are expected to help ARL libraries 
justify their growing budgets due to the great expense of electronic and networked 
services. These measures can help answer such questions as: How much are 
research libraries spending for electronic resources collectively and how much on 
average? How do expenditures for electronic resources compare across several 
research libraries? 












We have not included the cost of the technical staff and their training, the 
networking and equipment to provide access to the electronic resources as well as 
the time of all the staff involved. This will have to be addressed in the future. 

General Introduction to C1-C3 

T he report should include expenditures for electronic indexes and refer- 
ence tools, electronic full-text periodical collections and electronic 
journal back-files, and online searches of remote databases — whether accessed 
remotely or installed locally from CD-ROM, magnetic tapes, etc. The report 
should also include expenditures for materials purchased jointly with other institu- 
tions if such expenditures can be separated from other charges for joint services, 
fees paid to bibliographic utilities if the portion paid for computer files and search 
services can be separately counted, and equipment costs when they are inseparably 
bundled into the price of the information product. 

Expenditures for bibliographic utilities, networks, and consortia that are 
unrelated to end-user database access should be reported in C4, not in Cl through 
C3. 
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Expenditures for 
Networked Resources and 
Related Infrastructure 



□ Cl Cost of electronic full-text journals 

□ C2 Cost of electronic reference sources 

□ C3 Cost of electronic books 

□ C4 Library expenditures for bibliographic utilities, 
networks, and consortia 

□ C5 External expenditures for bibliographic utilities, 
networks, and consortia 




Cost of electronic full-text journals 

















Definition: Expenditures for electronic full-text journal subscriptions 
that the library provides to its users. Include both initial purchase cost, member- 
ship fees (such as JSTOR) as well as annual access and service fees paid directly or 
through consortia arrangements. 

Rationale: This statistic, cost of electronic full-text journals, was 
developed by ARL to find out how much libraries are spending on electronic full- 
text journals and how new forms of electronic journals are replacing traditional 
journals and scholarly publications. It also indicates the extent of budget alloca- 
tions for electronic resources. Furthermore, this statistic allows libraries to calcu- 
late unit costs of e-journals after collecting Cl and R1 statistics, and thus aids 
libraries in deciding how effectively they are serving their potential and intended 
audiences, and in benchmarking with the other institutions. 

Implementation 

□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly) 

□ Procedures: For the definition of electronic full-text journals, please refer to the 
definition of Rl. Current library accounting systems do not support coding of 
materials expenditures by the categories used in the manual. Therefore, it may 
become necessary to create an in-house spreadsheet or database file to keep 
track of cost information according to the types of resources (e-journals, refer- 
ence databases, and e-books). Preferably a single file will contain contract 
information (duration, cost), updated title counts, and reported usage statistics. 
Significant coordination is required for setting up the structure of the file, but in 
the long run may streamline many aspects of the management of electronic 
licensed materials. 

1. Gather reports and invoices related to electronic databases and resources 
for the reporting period. These documents are typically handled by the 
library’s accounting office. 

2. If you have not done so, organize the data using the sample worksheet in 

Appendix B, Figure B.l. / , 




BEST COPY AVAILABLE 



3. Even if a licensing contract or consortium arrangement period is different 
from the reporting period, use the annual licensing fee to calculate the 
statistics. 

4. If a fee is paid to a consortium or other joint arrangement, include the 
amount. In the case where a fee is paid for an aggregate service and the 
service contains different categories of resources (full-text journals and 
reference sources) as a bundle, use an estimate based on expected or 
historical use, or list prices. 

5. Note any major commitments (such as JSTOR one-time costs) that do not 
occur year to year and that significantly influence the reported amount. 

Special considerations: Whereas the patron accessible resource 
counts reflect the extensiveness of electronic resources at a given point in time 
(most likely at the end of the reporting period, be it a month or a year), the cost 
figures cover a period of time. Ideally the amount of money spent reflects the 
number and extensiveness of resources. Prorating licensing fees addresses part of 
the problem of matching the resources with the money spent, but can be very time 
consuming. You can report the annual amount paid without prorating on the basis 
that over the years the figures will even out. 

Some electronic full-text journals come either as a free service with a print 
subscription or as part of a print-plus-online-access subscription (the library pays 
extra for electronic access). In the first case, the problem is whether or not to post 
any amount for the cost of electronic access. In the latter case, the question is how 
much of the cost can be attributed to electronic access. 
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Cost of electronic reference sources 




Definition: Expenditures for electronic reference sources and aggre- 

gate services that the library provides to users either through individual licensing 
contracts with content providers or through consortia or other arrangements where 
the library pays some fees. These fees include both annual access fees and other 
service costs paid to the vendor directly or through consortial arrangements. 
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Rationale: This statistic, cost of electronic reference sources, was 
developed by ARL to determine how much libraries are spending on electronic 
reference sources and how new forms of electronic reference sources are replacing 
traditional reference materials. It also gives insight into shifts in budget alloca- 
tions from print to electronic materials, or new allocations exclusively for electronic 
materials. Furthermore, this statistic allows libraries to calculate unit costs of 
electronic reference sources after collecting C2 and R2 figures. This figure assists 
libraries in making decisions about how effectively they are serving their potential 
and intended audience, and in benchmarking with other institutions. 

Implementation 

n Collected by: Local 

n Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly) x 
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□ Procedures: For the definition of electronic reference sources, please refer to the 
definition of R2. For libraries that do not have acquisitions systems which 
support coding of materials expenditures by the categories used in the manual, it 
may be necessary to create an in-house spreadsheet or database file to keep track 
of cost information according to the types of resources (e-journals, reference 
databases, and e-books). Preferably a single file will contain contract informa- 
tion (duration, cost), updated title counts, and reported usage statistics. 

1. Gather reports and invoices related to electronic databases and resources 
for the reporting period. These documents are typically handled by the 
library's accounting office. 

2. If you have not done so, organize the data using the sample worksheet in 
Appendix B, Figure B.l. 

3. Even if a licensing contract or consortium arrangement period is different 
from the reporting period, use the annual licensing fee to calculate the 
statistics. 

4. If a fee is paid to a consortium or through other joint arrangement, include 
the amount. If a fee is paid for an aggregate service and the service 
contains different categories of resources (full-text journals and reference 
sources) as a bundle, use an estimate based on expected or historical use, or 
list prices. 

5. In the comments field of the sample worksheet (Appendix B, Figure B.l), 
report any major commitments that do not occur year to year and that 
significantly influence the reported amount. 

Special considerations: Whereas the patron accessible resource 
counts reflect the extensiveness of electronic resources at a given point in time 
(most likely at the end of reporting period, be it a month or a year), the cost figures 
cover a period of time. Ideally the amount of money spent reflects the number and 
extensiveness of resources. Prorating licensing fees addresses part of the problem 
of matching the resources with the money spent, but can be very time consuming. 
You can report the annual amount paid without prorating on the basis that over the 
years the figures will even out. 

Cost of electronic books 








Definition: Expenditures for electronic full-text monographs that the 
library offers to its users. Include both initial purchase costs and membership fees 
as well as annual access and service fees paid directly or through consortia arrange- 
ments. 

Rationale: This statistic, cost of electronic books, was developed by 
ARL to determine how much libraries were spending on electronic books. It also 
gives an idea about the extent of budget allocations for electronic resources. Fur- 
thermore, this statistic allows libraries to calculate unit costs of e-books after 
collecting C3 and R3 statistics, aids them in determining how effectively they are 
serving their potential and intended audiences, and assists them in benchmarking 
with other institutions. 



Implementation 

□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(monthly, quarterly) 

□ Procedures: For the definition of electronic books, please refer to the definition 
of R3. Current library accounting systems generally do not support coding of 
materials expenditure by the categories used in the manual. Therefore, it may 
become necessary to create an in-house spreadsheet or database file to keep 
track of cost information according to the types of resources (e-journals, refer- 
ence databases, and e-books). Preferably a single file will contain contract 
information (duration, cost), updated title counts, and reported usage statistics. 

1. Gather reports and invoices related to electronic databases and resources 
for the reporting period. These documents are typically handled by the 
library's accounting office. You may also need to review circulation records 
to verify the accuracy of invoices if additional per-use fees are paid (royalty 
on use, as with E-reserves). 

2. If you have not done so, organize the data using the sample worksheet in 
Appendix B, Figure B.l. 

3. Even if a licensing contract or consortium arrangement period is different 
from the reporting period, use the annual licensing fee to calculate the 
statistics. 

4. If a fee is paid to a consortium or other joint arrangement, include the 
amount. 

5. Note any major commitments (such as netLibrary purchase costs) that do 
not occur year to year and that significantly influence the reported amount. 

Special considerations; Whereas the patron accessible resource 
counts reflect the extensiveness of electronic resources at a given point in time 
(most likely at the end of the reporting period, be it a month or a year), the cost 
figures cover a period of time. Ideally the amount of money spent reflects the 
number and extensiveness of resources. Prorating licensing fees addresses part of 
the problem of matching the resources with the money spent, but it can be very 
time consuming. You can report the annual amount paid without prorating with 
the rationale that over the years the figures will even out. 

Traditionally books are purchased on a one-time payment in exchange for 
permanent ownership by the library. However, with regard to electronic books, it 
appears that some arrangements allow libraries to subscribe to an e-book collection 
at a predetermined fee and for a predetermined interval of time. We are concerned 
with the format of the material, not the subscription or payment arrangement. 
These materials should be counted as books, not serial publications. 

Related issues: In many instances, the physical form of the material 
(print, electronic) may change the nature of the object. An electronic book is a 
good example. With enhancements such as full-text searching (although print 
books too have some search capability through tables of contents and indexes), 
electronic books support new forms of searching not present in print. 
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© Library expenditures for bibliographic 

UTILITIES, NETWORKS AND CONSORTIA 

Definition: Expenditures paid by the library for services provided by 
national, regional, and local bibliographic utilities, networks, and consortia such as 
OCLC, KLGyexcluding fees paid for user database access and subscriptions y which 
should be reported in Cl through C3. 

Rationale: This statistic is based on the ARL Supplementary Statistics 
Survey. It was developed by ARL to determine how much money libraries spend 
for bibliographic utilities, networks, and consortia. Because individual libraries 
often have to deal with special provisions and funding issues related to contracts, 
this statistic may not lend itself to comparability among ARL member libraries. 
Nevertheless, it represents an attempt to keep track of the financial relationships 
between bibliographic utilities and libraries. Although this may provide very 
limited comparability, it is an estimate of the cost of bibliographic utilities, net- 
works, and consortia. 
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Implementation 

□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more fre- 
quent (e.g., monthly, quarterly) 

□ Procedures: 

1. Gather reports and invoices with bibliographic utilities, networks, and 
consortia of which the library is a member for the whole or part of the 
reporting period. These documents are typically handled by the library’s 
accounting office. 

2. Identify only those expenditures paid to the bibliographic utilities, net- 
works, and consortia for membership, maintenance, and other infrastruc- 
ture. Do not include expenditures that are directly attributable to access of 
electronic resources. Those expenditures should be included in Cl through 
C3. 

For instance, if your library paid a total of S100,000 to OCLC for its 
various services and your best guess of electronic database access portion of 
the services is 80%, then you should report S80,000 for C2 and the re- 
maining S20,000 for C4. 

3. Even if a membership or consortium period is different from the reporting 
period, use the amount of the membership or consortium agreement. 

4. Use the sample form in Appendix B, Figure B.2 to compile the expendi- 
tures. 

Special considerations: Prorating can be time consuming. Consor- 
tia or other memberships may bring additional benefits, such as subscriptions, 
training or preferential pricing for acquisition of materials. It may be difficult to 
separate pure membership fees from value-added services of membership (e.g., 
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original catalog credits from OCLC that may be used to offset costs of databases, 
purchase of catalog records, etc.). Report the annual amount paid without prorat- 
ing with the rationale that over the years the figures will even out. 




External expenditures for bibliographic 

UTILITIES, NETWORKS, AND CONSORTIA 



Definition: Expenditures paid by external agencies, such as state 
government agencies, on the library’s behalf for access to computer files, electronic 
serials, or search services through a centrally funded system or consortial arrange- 
ments. Examples include state- (or province-) supported networks such as VIVA 
(Virginia), CNSLP (Canadian National Site Licensing Project), and the University 
of California’s California Digital Library Expenditure. 

Rationale: Like statistic C4, this statistic is based on the ARL Supple- 
mentary Statistics Survey. It was developed by ARL to determine how much money 
is spent for bibliographic utilities, networks, and consortia on libraries’ behalf for 
access to computer files, serials, and/or services through consortial arrangements. 
Because of contractual issues, this statistic may provide little comparability among 
ARL member libraries. Nevertheless, it can give ARL members an estimate of the 
external costs of bibliographic utilities, networks, and consortia. 

Implementation 

□ Collected by: Local and external bodies such as regional and academic consortia 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly). 

□ Procedures: 

1. Gather reports and invoices with bibliographic utilities, networks, and 
consortia that are related to electronic databases and resources for the 
reporting period. These documents are typically handled by the library’s 
accounting office. However, they can be maintained outside the organiza- 
tion and, in some instances, may only be provided to libraries upon de- 
mand. 

2. Find out how much of the central funding is attributable to your library. 

For example, if your library contributes a total of S60,000 over a period of 
three years to a state consortium that has a matching contribution of 
$120,000 for the same period, the amount to report as C5 for a given year 
during the three-year period will be $40,000 ($120,000 x 1/3). The 
library’s contribution ($60,000) has to be divided annually and posted in 
Cl through C3. 

If the specific dollar amount is not known, but the total student FTE (full- 
time equivalent) for the consortium and the amount spent for the academic 
members are known, divide the overall amount spent by your institution’s 
share of the total student FTE. Alternatively, if the consortium is com- 
prised of different types of institutions (academic, public, or corporate), 
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but the library has information about the portion of its own use among the 
consortium participants, multiply the total amount by the percentage of 
known (or estimated) usage rate. 

3. As a last resort, consult with a staff member overseeing the consortium or 
the central funding system to get an estimate of the portion of the central 
funding that is attributable to the library. Please make a note of this in the 
comments field in the sample worksheet (Appendix B, Figure B.3). 

4. Use the sample form in Appendix B Figure B.3 to compile the expendi- 
tures. 
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Statistics Related to Library Digitization Activities 
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C omprised of resource and use measures, the digital collection measures 
attempt to describe where libraries are in creating and making available 
local (perhaps unique) content that may not have been previously accessible. Such 
collections can attract students and faculty to your university and thereby enhance 
the institution’s reputation. As more libraries digitize resources, more users will be 
able to retrieve those unique resources at anytime and from anywhere. Digital 
library projects, as well as other network resources and services, also will serve 
increasing numbers of students taking courses online. 






Collecting library digitization measures may provide an opportunity for 
benchmarking and may encourage libraries to devote more time and allocate more 
resources to this worthwhile endeavor. It should be noted that these statistics 
represent a very early attempt to measure digitization of resources; as time passes 
and the technology advances, some of the definitions and procedures may need to 
be revisited and modified. During the field-testing it was reported that storing 
and maintaining digitized resources had been an issue. The unavailability of an 
appropriate infrastructure in some institutions meant that the project did not 
include statistics related to library digitization projects. 
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Libraries archive the scholarly output of their institutions - theses and 
dissertations - in both paper and digital form. Digital collections also provide new 
opportunities with faculty to archive research results. These statistics, although 
preliminary, form a basis for tracking these issues. 



i □ D1 Size of library digital collection | 

I □ D2 Use of library digital collection | 

! □ D3 Cost of digital collection construction and I 

I management j 

! i 

I I 



Library 

Digitization 

Activities 



Size of Library digital collection 

Definition: Library digital collection refers to digital materials (texts, 
images, and audio-visuals) created in or converted from different formats (e.g., 
paper, microfilm, tapes, etc.) by the library and made available to users electroni- 
cally. This includes electronic theses and dissertations (ETDs), special collections 
materials, maps, sound recordings, films, and other digital materials that are not 
purchased or acquired from outside through individual or consortial licensing 
agreements. It includes the number of titles and size (in gigabytes) by sub-catego- 
ries (ETD, visual materials, texts, multimedia), and as an aggregate at the end of 
the reporting period. It also includes the number of items (titles) added during the 
reporting period. 
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The types of formats in Appendix B, Figure B.4, refer to original formats 
rather than the digitized outputs. Examples of visual materials include photos, 
maps, and postcards. Examples of text include books, journal articles and pam- 
phlets. Examples of multimedia include audio, video, and other interactive materi- 
als. However, this statistic does not include any back up copies or mirror sites 
because items should be counted only once. 

Rationale: Collecting library digitization measures may provide an 
opportunity for benchmarking in terms of file sizes for the resources that have 
been digitized. Moreover, the statistic can demonstrate that libraries are not 
merely brokers of external information resources, but also producers of information 
content and useful finding aids. 

This statistic provides information on the extent of digital library projects, 
the life cycle of such projects, and the “virtual space” requirements of such collec- 
tions. 

Implementation 

□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly) 

□ Procedures: For cases in which multiple digital formats (derivatives) were 
produced from an item, count it only once based on the type of item that was 
digitized. For example, if a 100-page book was digitized in 100 TIFF files, each 
containing a page, a 100-page PDF file, and 10 PDF files (one PDF file for each 
of 10 chapters), count it as a single text with 100 pages. If a derivative item 
was used as the source, do not count the outputs. But in the total size (in 
gigabytes) include all versions of derivatives. 

1. Designate a staff member to coordinate the collection of this statistic. The 
person should be well aware of library digital collection activities. 

2. Identify library staff in charge of various digital library projects and 
initiatives. 

3. It is necessary to conduct an inventory of digital material stock using the 
sample tally worksheet in Appendix B, Figure B.6 if it has not been done 
already. If this inventory information is already available, enter it into the 
worksheet. When the inventory is completed, summarize the information 
using the sample worksheet in Appendix B, Figure B.4. Add additional 
categories if necessary. 

4. After obtaining the inventory information, ask staff members to keep track 
of additional output regularly using the sample tally worksheet in Appen- 
dix B, Figure B.6. 

5. At the end of the reporting period, collect the worksheets and calculate the 
total production during the reporting period using the worksheet in 
Appendix B, Figure B.5. Add additional categories if necessary. 



Related issues: Realistically^ each digital collection is unique in terms 
of the production process, the way it is intended to be used, its focus, and mainte- 
nance. It is important to use appropriate units of measurement to describe the 
overall size and extensiveness of the whole collection. 

Because of the wide variations of the types and features of digital collections 
constructed at ARL institutions, this statistic may be more useful locally than for 
comparison across ARL member libraries. Benchmarking may, however, be possible 
from the data collected to produce some qualitative and quantitative indicators as 
to the extent of digital library collection activities and different emphases across 
the ARL membership. 
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Use of library digital collection 



Definition: Number of times library digital collection titles and 
physical files were accessed and the number of searches (queries) conducted (if 
there is such a capability) during the reporting period. 



mm D 



Rationale: Each digital collection is unique in terms of its focus, 
production process, and the ways it is intended to be used and maintained. There- 
fore, because of the wide variations of the types and features of these library 
collections constructed at different ARL institutions, this statistic needs to be 
collected and used locally instead of across ARL member libraries. Nevertheless, 
this statistic has the potential to produce some qualitative and quantitative indica- 
tors as to how these collections are being used and serving the intended user 
community’s needs. 
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Implementation 



□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more frequent 
(e.g., monthly, quarterly) 



□ Procedures: 

1. Designate a staff member to coordinate the collection of this statistic. The 
person should be well versed in the use of web log software and/or statis- 
tics provided by the software. This person will act as a liaison for staff 
members who are responsible for managing library digital collections. 
Obtaining the statistic may require some level of programming (e.g., Unix 
scripting and SC^L). 

2. Items accessed can be collected in various ways, and depending on your 
library’s environment, your library may need to collect different access 
statistics. 

Although you are asked to collect both title access and physical file access, 
if it takes too much time and effort to collect the title access, report the 
physical file access count only. For example, a book can be digitized and 
made into 10 PDF files, each containing a chapter, for access. Suppose a 
user viewed five PDF files out of 10. In this case, you will have five 
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physical item accesses and one title access. Usually it is easier to have a 
physical item access count, while it takes custom programming to compute 
the title access count as most off-the-shelf web traffic software packages 
do not provide this. 

Do not report web page hits. Instead, count how many times the digitized 
items were accessed (the exact name for item access may vary depending on 
the type of web traffic analysis software being used in the library). 

If a search capability is a feature of a library digital collection, the total 
number of searches submitted needs to be collected. A search represents 
an explicit user request for specific information in a database and is ex- 
pressed usually in the form of word strings. Clicks on web page buttons, 
such as “Next” and “Previous,” do not count as user searches. 

You might want to install web traffic analysis software (e.g., WebTrends, 
Web Tracks) on the library web servers housing library digital collection 
materials, if the web servers do not have such software already. You might 
want to consider installing a trial version that gives between 30-180 days 
of free trial. 

Read the description of reported statistics carefully and make sure that the 
software provides what you want. 

3. If continuous collection of use statistics is not possible or desirable, select a 
typical week (or month) to run a sample study. Be sure to vary the specific 
week (or month) chosen over the course of a year or from year to year to 
account for seasonal fluctuations. Extrapolate based on the sample data. 

4. At the end of the report period, use the log analysis report to calculate the 
number of accesses to library digital collection items. Use the sample 
report in Appendix B, Figure B.7, to organize the information. 

Special considerations: To the extent possible, exclude accesses by 
web search spiders. Also, do not include accesses to auxiliary (or incidental) items 
that are not part of the library digital collection content (.gif buttons and image 
maps for navigation). Note the method used and include a description of any 
filtering done. 

Related issues: This statistic needs to be collected and used locally 

instead of across ARL member libraries because of the wide variations of the types 
and features of digital collections constructed at ARL institutions. 




Cost of digital collection 

CONSTRUCTION AND MANAGEMENT 



Definition: Annual direct costs (personnel, equipment, software, 
contracted services and similar items) spent to create digital materials (texts, 
images, and multimedia) or to convert existing materials into digital form for the 
purpose of making them electronically available to users. Include expenditures 
related to digitization, OCR, editorial, creation of markup texts, preparation of 
J metadata for access to digitized materials, data storage, and copyright clearance. 



Exclude expenditures for information resources purchased or acquired from outside 
the institution through individual or consortial licensing agreements. 

Rationale: The cost of each digital collection construction may vary 
significantly, depending on the size of the collection, conditions of the sources 
before digitizing, available infrastructure, staff allocation, timeline, and administra- 
tive support. This statistic should be collected and used locally instead of across 
ARL member libraries because of the wide variability among these library collec- 
tions constructed at different ARL institutions. Nevertheless, this statistic has the 
potential to provide quantitative indicators as to how costly these efforts are, how 
much resource allocation (i.e., budget allocation, staffing, infrastructure, etc.) is 
needed, and how well they serve the intended user community's needs (e.g., to 
account for internal and external costs to construct and manage digital collections 
at ARL libraries). 

Implementation 

□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more fre- 
quent (e.g., monthly, quarterly) 

□ Procedures: 

1. Designate a staff member to coordinate the collection of this statistic. 

2. Direct the designated staff member to contact library staff members who 
are in charge of digital collection projects. Ask all library staff members 
involved in any digital collection projects as part of their official responsi- 
bilities to fill out the worksheet in Appendix B, Figure B.8, for the report- 
ing period. Ask them to estimate how much of their time was spent on 
planning, implementing, and managing digital collection projects. This 
information will be entered in the worksheet as FTE. A further break- 
down of activities may be necessary if the library wants to have more 
detailed information on the distribution of efforts. 

Note that annual salary should not be asked of the staff members filling 
out the worksheet and should not include fringe benefits. When all the 
worksheets are collected, the salary information will be obtained from the 
library accounting or personnel department. Direct staff cost will then be 
calculated. 

The personnel cost should also include wages paid to non-salaried staff, 
including student and other hourly workers. 

3. Cost of equipment should be amortized. For example, if a $3,000 scanner 
was purchased at the beginning of the reporting year and has a deprecia- 
tion period of three years, register $1,000 as the equipment cost. Costs of 
software should be reflected in full amounts based on the time of the 
purchase. 

4. If a subcontracting period is different from the reporting period, prorate 
the amount for the reporting period. If the payment is based on percent to 
completion, include only the amount that belongs to the reporting period. 

5. Use the sample worksheet in Appendix B, Figure B.9, to calculate the total 
cost. 
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Related issues: This statistic needs to be collected and used 

locally instead of across ARL member libraries because of the wide variations of 
the types and features of digital collections constructed at ARL institutions. 
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Analysis of Suogested Performance Measures 




T he overall rationale for the performance measures in this study is to 
provide a means for measuring the proportion of services delivered 
through traditional channels relative to analogous services delivered through 
electronic channels. These measures will help document trends in service delivery 
for the purpose of allocating staff and development resources as well as identify 
trends for strategic planning of service delivery. 

□ PI Percentage of electronic reference transactions of total 
reference 

□ P2 Percentage of virtual library visits of all library visits 
□ P3 Percentage of electronic books to all monographs 



Performance 

Measures 
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O Percentage of Electronic Reference 
Transactions of Total Reference 



© 



Definition: Percentage of annual electronic reference transactions to 
total reference transactions. An electronic reference transaction must include a 
question either received electronically (e.g., via e-mail, WWW form, etc.) or 
responded to electronically. Count excludes phone and fax traffic unless either the 
question or answer transaction occurs via the described manner. It includes the 
counts accrued from participation in any local and national projects, such as 
DigiRef and the Library of Congress’s CDRS (Collaborative Digital Reference 
Service). 

Total reference = Traditional reference counts (include face-to-face reference 
transactions, telephone and fax reference counts) + electronic reference transaction 
counts. 







U1(p.11) 



Total reference transactions 



• 100 






Rationale: The purpose of having a networked environment is to 
promote connectedness. This measure provides an indication of a changing library 
environment. While in the traditional library environment reference transactions 
were handled mainly through non-electronic means, in the current environment 
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reference transactions can be handled via various electronic means over the 
Internet. By having this as a measure, libraries are able to track the development 
of a new and emerging library service and have a number that fully represents 
reference activities. This measure may indicate how often various electronic 
applications are used in any given period and also assist decision-makers in reallo- 
cating resources. Moreover, this performance measure will give administrators 
trend data on how network services are being used and this data can then be used 
for future planning. 

Implementation 

□ Collected by: Local 

□ Frequency: Reported annually, although internal reporting may be more 

frequent (monthly, quarterly) 

□ Procedures: 

1. If continuous collection of this statistic is not possible or desirable, select a 
typical week (or month) to run a sample study. It is recommended that you 
sample a week in a different month or several months to account for 
seasonal fluctuations. Extrapolate based on the sample data. 

2. Designate a staff member to coordinate the collection of this measure. Key 
tasks include distributing a daily tally sheet, collecting the daily tally sheet, 
adding each day^s total to a weekly figure, and being available to respond to 
data collection problems should they arise. 

3. For electronic transactions, use the count obtained by following the proce- 
dures for U1 (p. 11). 

4. Total the overall number of transactions. 

5. Divide the number of electronic reference transactions by the total number 
of transactions. 

6. Multiply by 100. 

7. Indicate and describe any additional methods used outside of this defini- 
tion and these guidelines. 

Special considerations; Count the number of transactions, not the 
number of questions. That is, if one request is emailed with three questions, it 
should be counted as one transaction, not three. 




Percentage of Virtual Library Visits 
OF All Library Visits 



Definition; Number of virtual library visits out of all library visits. 

A virtual library visit is when a user visits the library's website or catalog for 
any length of time or for any purpose from outside the physical plant of the library, 
regardless of the number of pages or items viewed or requested. The term “virtual 
visit” excludes in-library visits where a patron or a staff member uses electronic 



resources. If a user looked at 16 pages and 54 graphic images while at a website, 
that user registers one visit on the web server. A visit is usually determined by a 
user’s IP address. Due to various server management issues and differing software, 
this measure is an estimate of the visits to the library site. 

All library visits is the total of the number of virtual library visits plus the 
number of physical visits to the library including branches. 







U5(p. 16) 



Total Lbrary Visits 



*** 100 




Rationale: People accessing the website or catalog from outside the 
library will reflect interest in library services. The idea of having network services 
is to expand the reach of libraries beyond their physical boundaries, and this 
performance measure can provide information about how far network services are 
reaching. This figure will also show the use of the library outside the regular place 
of business, which will be a more accurate depiction of library use. Having this 
measure is important to show the continued relevance of library service if physical 
attendance figures decrease. 

Implementation 
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□ Collected by: Local and/or vendors 

□ Frequency: Reported annually, although internal reporting may be more 

frequent (monthly, quarterly) 

□ Procedures: 

1. Obtain the virtual library visits count using the procedures for U5 (virtual 
library visits). 

2. Obtain physical attendance count from turnstile counts or swipe card 
records. To the extent possible, collect comprehensive data from all library 
branches. 

3. Combine the virtual visit count and the physical attendance count. 

4. Divide the number of virtual library visits by the total library visits. 

5. Multiply by 100. For example, a library had 1,000 external virtual visits 
and 9,000 physical visits for a total visit composite measure of 10,000. 
1,000 virtual visits divided by 10,000 total visits equals .10 (or 10%). 






© Percentage of Electronic Books to 
All Monographs 

Definition; Percentage of the number of electronic books available to 
users (through either an individual licensing contract or other consortial arrange- 
ments) to all the library’s monographs. 









Rationale: Networking technology in libraries continues to improve, 
thereby increasing user access to electronic counterparts of some traditional 
sources. In the mid 90s, networking and resource sharing technologies facilitated 
print and e-book access through library networks. This performance measure 
attempts to document the degree of expansion of e-books to all monographs. 
Libraries should use caution while collecting this measure because the definition of 
e-books is itself still evolving. This statistic is an early attempt to keep track of 
this type of source that is becoming increasingly available. 

Implementation 

□ Collected by: Local and vendors 

□ Frequency: Reported annually, although internal reporting may be more fre- 
quent (monthly, quarterly) 

□ Procedures: 

1. Identify all types of monographic materials. Use in-house record-keeping 
sources and other library sources to determine the number of all mono- 
graphs, including electronic books, non-electronic books, and other mono- 
graphic materials. 

2. Identify electronic book types, including electronic books and electronic 
full-text aggregate services, using the sources in step 1 of the procedures 
for R3. 

3. Count individual electronic book titles. Record the number of individual 
electronic books from the spreadsheet or record the number from another 
source. 

4. Exclude electronic reference books, i.e., publicly available electronic books 
that are accessed for free. 

5. Calculate the total number of all monographs, including electronic books, 
non-electronic books, and other monographic materials. 

6. Divide the number of electronic books by the number of all library mono- 
graphs (electronic and non-electronic monographs). 

7. Multiply by 100. 

8. Indicate and describe any additional methods used outside of this defini- 
tion and guidelines. 
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Improving Networked Statistics 



With the ever-increasing portion of library collections* dollars committed to 
networked services, there is a pressing need to better understand the impact from 
the increase of such services and supporting technology. To begin overcoming the 
relatively little that is known about how these services are used, who uses them, 
and what impact these services have, the statistics and performance measures 
offered in this manual provide a start. 

As reflected by the interest and efforts of the many ARL libraries that 
participated in the E-Metrics project, the development of library networked 
statistics and performance measures continues to receive increased attention and 
support. There is broad recognition of the need for network statistics and perfor- 
mance measures that: 

□ Assist libraries in making a strong case for support of technology and 
information infrastructure by documenting their Internet-based services 
and resources; 

□ Assist libraries in demonstrating the use of digital collections in order to 
make a case for continued collection development and support; 

□ Allow libraries to effectively compare themselves to others in terms of 
Internet-based collection and service development, costs, provision of 
services, connectivity, and use; 

□ Allow libraries to measure and track internal changes to library operations 
as well as uses and users of library resources and services; 

□ Enable library directors and administrative library agencies to compete for 
resources with other organizations and/or departments by documenting the 
range, extent, and impact of library-provided networked services; 

□ Facilitate the expansion from traditional library use measures such as 
circulation, reference transactions, interlibrary loans, etc,, to include 
network measures that describe the nature and use of library-based net- 
work activities and resources; 

□ Provide a decision-making framework for library staff, managers, and 
administrators to determine resource allocation strategies and meet other 
management needs; 

□ Provide a means through which to measure the quality of library services 
and resources in the networked environment. 

These and other factors point to the overall importance of the development, 
collection, and reporting of library network statistics and performance measures to 
facilitate collections decisions, cost analysis, justification of services, services 
planning and evaluation, and a host of other activities. It is hoped that the statis- 
tics and measures developed herein help fill many of the needs faced by academic 
and research libraries. 

However, there are a number of issues and challenges that affect the library's 
ability to collect statistics and measures to describe its electronic resources and 
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services. Some academic and research libraries possess inadequate resources, 
staffing, and expertise to collect, manage, and report the data related to describing 
networked services. For these libraries, some organizational development and 
commitment to collecting and using these data may be necessary to take advantage 
of the measurement tools and techniques outlined in this manual. The discussion 
of measurement issues in Parts 1 through 3 of the E-Metrics Phase II Report 
<http://www.arl.org/stats/newmeas/emetrics/> can assist libraries in better under- 
standing why such measurement is essential. 
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Appendix A: List of Statistics Considered 



Categories 


E-Metrics vl.O (Initiai List) 


E-Metrics ?.U (Revised List) 


E-Metrics v.2.0 (Field Test List) 


Resources 


□Number of electronic full-text 
journals (hosted by library) 
□Number of electronic full-text 
Journals (through subscription) 
□Number of librarians providing 
electronic reference 
□ Number of public access 
workstations 


□Number of electronic full-text 
periodicals (hosted by library) 
□Number of electronic full-text 
periodicals (through institutional 
subscription) 

□Number of electronic full-text 
periodicals (through consortia and 
other arrangements) 

□ Number of electronic reference 
databases (through institutional 
subscription) 

□Number of electronic reference 
databases (through consortia and 
other arrangements) 

□ Number of electronic books 

□ Number of staff providing 
electronic reference 
□Number of public access 
workstations 


□ Number of electronic full-text 
journals (institutional) 

□ Number of electronic full-text 
journals (consortia) 

□ Number of electronic reference 
sources (institutional) 

□ Number of electronic reference 
sources (consortia) 

□ Number of electronic books 
(institutional) 

□ Number of electronic books 
(consortia) 


Use 


□ Logins (sessions) 

□Queries (searches) 
□Turn-aways (requests exceed 
simultaneous user limit) 

□ Items examined (viewed, 
downloaded, emailed, printed) 
□Total user connection time to 
vendor databases 

□Virtual visits to networked 
library resources 

□ Electronic reference 
transactions 

□ Number of people 
participated in user instruction 
on electronic resources 


□Number of logins (sessions) to 
networked library resources 

□ Electronic reference transactions 
□Number of Logins (sessions) to 
electronic databases 
□Queries (searches) 

□Total connection time to 
electronic databases 

□ Items examined (viewed, 
downloaded, emailed, printed) to 
electronic databases 
□Turn-aways (requests exceed 
simultaneous user limit) 

□Number of people participated 
in user instruction on electronic 
resources and services 


□ Number of electronic reference 
transactions 

□ Number of logins (sessions) to 
electronic databases 

□ Number of queries (searches) in 
electronic databases 

□ Items examined in electronic 
databases 


Cost 


□Cost of electronic database 
subscriptions 

□Cost per items examined 
(subscribed databases) 


□Cost of electronic files (one- 
time/monographic purchase) 
□Cost of electronic full-text 
periodicals subscriptions 
□Cost of electronic reference 
databases subscription 
□ Library contribution to consortia 
for electronic databases 


□Cost of electronic full-text journals 
□Cost of electronic reference 
sources 

□Cost of electronic books 
□Library expenditures for bib. 
utilities, networks, and consortia 
□ External expenditures for bib. 
utilities, networks, and consortia 


Local Dioital 
CollectioD 


□Cost of internal digital 
collection construction 


□Cost of internal digital collection 
construction 


□ Size of library digital collection 
□Use of library digital collection 
□Cost of digital collection 
construction and management 


Performauce 

Measures 


□Percentage of electronic 
reference transactions of total 
reference 

□Percentage of electronic 
materials use of total library 
materials use 

□Percentage of remote library 
visits of all library visits 
□Ratio of public access 
workstations to university 
population (number of faculty, 
staff, and students) 


□Percentage of electronic 
reference transactions of total 
reference 

□Percentage of electronic 
materials use of total library 
materials use 

□ Percentage of remote library 
visits of all library visits 
□Percentage of electronic titles to 
all periodicals 

□Percentage of electronic books 
to all monographs 
□Ratio of public access 
workstations to university 
population 

□Cost per items examined in 
individually subscribed databases 


□Percentage of electronic reference 
transactions of total reference 
□Percentage of electronic materials 
use of total library materials use 
□Percentage of remote library visits 
of all library visits 
□Percentage of electronic books to 
all monographs 
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BEST COPY AVAILABLE 



Appendix B: Forms 




Forms List 



Figpre B.l Sample Electronic Resodrce Cost Report Form 
Figpre B.2 Sample Consortia Expenditpre Report form 
Figpre B.3 Sample Consortia Fending Report Form 
Figpre B.4 Sample Lirrary Digital Collection Inventory Report Form 
Figpre B.5 Sample Digital Collection Iitems Added Report Form 
Figpre B.6 Sample Lirrary Digital Collection Report Form 
Figpre B.7 Sample Digital Collection Access Report Form 
Figpre B.8 Sample Digital Collection Cost Report Form-Personnel 
Figpre B.9 Sample Digital Collection Cost Report Form 
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Figure B.1 (for C1-C3) 






Sample Electronic Resource Cost Report Form 






Reporting Period: 
Name of library: 



Resource/ Consortium Name and Type 
Full-teit journals (!) 
Reference sources (2) 
Electronic books (3) 


Cost 


Comments 










































































Sub Total (I) 






Sub Total (2) 






Sub Total (3) 






Grand Total (1+2+3) 
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Sample Consortia Expenditure Report Form 






Reporting Period: 
Name of library: 



Consortium Name 


Amount 


Comments 

































































(5^ 
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Figure B.3 (for C5) 



Sample Consortia Funding Report Form 






Reporting Period: 
Name of library: 



Consorlium 

Name 


To(al Funding 
Amounl 


Amounl 
AKribulable 
to tbe 
Library 


Comments 
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Figure B.4 (for D1) 



Sample Library Digital Collection Inventory Report Form 






Reporting period: 
Name of library: 



ETDs 


Visual 

Materials 


Texts 


Audio/Video/Multime- 

dia 


Total 


Titles 

(1) 




Items 

('2) 




Titles 

(3) 




Titles 

(4) 




Titles 

(1+2+3+4) 




















Size (GB) 








& 



(s^ 







O 

ERIC 
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Figure B.5 (for D1) 



Sample Digital Collection Items Added Report Form 






Reporting period: 
Name of library: 





ETDs 


Visual 

Materials 


Texts 


Audio/YIdeo/Multiie- 

dia 


Total 


n 1 i 


Titles 

(1) 




Items 

(2) 




Titles 

(3) 




Titles 

(4) 




Titles 

(1+2+3+4) 






















Size (GB) 



























0 ) 



& 



& 







Figure B.6 (for D1) 



Sample Library Digital Collection Report Form 






Reporting Period: 
Name of library: 



Project 

Name 


Information Type 

1. ETD 

2. visual Materials 

3. Texts 

4. Audlo/VIdeo/MultImedIa 


Server 

Name 


Directory 

Location 


No. of 
Titles 


Size(GB) 
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Sample Digital Collection Access Report Form 



Reporting Period: 
Name of library: 



Project 

Name 


Serrer 

Name 


Director; 

LocatioD 


Title 

Access 

CotiDt 


Item 

Access 

CouDt 


Total 

Searches 


Commeots 
























































































& 



& 







Sample Digital Collection Cost Report Form - Personnel 



Reporting Period 

Name of library: ^ , 



Name 


PosilloD 


fAflouai Salary} 


FTE 


iSlaff Cost) 
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Figure B.9 (for D3) 



Sample Digital Collection Cost Report Form 



Reporting Period: 
Name of library: 



Project Name 


Expense Type 


Amount 












































Project (name,) Total 






Project (name) Total 






Library Total 
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